Mutual information derived tree structure in an adaptive pattern recognition system

ABSTRACT

AN ADAPTIVE PATTERN RECOGNITION SYSTEM IS PROVIDED WHICH CALCULATES THE MUTUAL INFORMATION PROVIDED BY PAIRS OF FEATURES EXTRACTED BY A FEATURE EXTRACTING DEVICE. THE RELATIVE MAGNITUDES OF MUTUAL INFORMATION ARE DETECTED SERIATIM AND A CLOSED LOOP AVOIDANCE MODULE PREVENTS FORMING A CLOSED LOOP, TO RETAIN A STATISTICAL TREE RELATIONSHIP. PATTERN LOGIC STORES THE SET OF PAIRS HAVING HIGHEST VALUES OF MUTUAL INFORMATION. THEN THE SYSTEM IS PREPARED TO OPERATE A RECOGNITION SYSTEM. THE INDIVIDUAL FEATURES ARE WEIGHTED, ACCORDING TO STATISTICAL ANALYSIS, BY ANALOGUE COMPUTERS. ALSO, THE PAIRS OF INFORMATION ARE GATED AND WEIGHTED FOR EACH PATTERN IN ACCORDANCE WITH STATISTICAL WEIGHTING PRINCIPLES. THE SUMMING NETWORK FOR A PLURALITY OF PATTERNS ARE COMPARED IN A MAXIMUM DETECTOR FOR ULTIMATE RECOGNITION OF THE MOST LIKELY PATTERN IDENTIFICATION.

O United States Patent 1 1 3,588,823

[72] Inven r Ch K- Chow 3,239,811 3/1966 Bonner 340/1463 Chappaqua; 3,275,985 9/1966 Dunn et al. 340/1463 M Yorktown Heights Primary Examiner-Maynard R. Wilbur [2|] Appl. No. 716,732

. ASSISHZH! Exammer- Leo H. Boudreau [22] Filed Man 1968 Atlorne s-Hanifin and Jancin and Graham S .10 ll [45] Patented June 28, 197! y [73] Assignee International Business Machines Corporation Armonk, N.Y.

ABSTRACT: An adaptive pattern recognition system is pro- [54] I MUTUAL INFORMATION DERIVED TREE vided \tviiich calculates the mutual information provided by STRUCTURE IN AN ADAPTIVE PATTERN palrs o eatures extracte by a teature extracting device. The RECOGNITION SYSTEM relative magnitudes of mutual information are detected 14 Claims 30 Drawing Figs seriat1m and a closed loop avoidance module prevents forming a closed loop, to retain a statistical tree relationship. Pattern LS. logic tores the et of pairs haying highest values of mutual in- 340/l formation. Then the system is prepared to operate as a recog- [S l 1 ll!!- nition ystem The individual features are weighted according [50] Field of Search .i 340/ 146.3, to statistical ana|ysisy by analogue computers Also the pairs 172-5 of information are gated and weighted for each pattern in accordance with statistical weighting principles. The summing [56] References cued networks for a plurality of patterns are compared in a max- UNTED STATES PATENTS imum detector for ultimate recognition of the most likely pat- 3,045,9ll 7/1962 Russell et al ..(340/146.3UX) tern identification.

k l 149 i l G a 19- START f A SAMPLE SAWTOOTH ,'COUNTER g 211 GENERATOR v I v 1 V I 1 E C F R E A l M P P Q A v I 3 1 N T o A i U C l T S C R R F R T C E F H l l L E l E R E E E S O 1 N s R s s o P G I 1- N S I L l COMBINATION OF Two 1 62 AND GATES i 7 11-16 w -c WEIGHTING COMPUTERS j Patented June 28, 1971 24 Sheets-Sheet l COMPAR SON SAWTOOTH GENERATOR START MEMOR ES SW TCHES FIG. 1A

TRANSFER SAMPLE COMPUTERS COMBINATION OF TWO AND GATES WEIGHTING COMPUTERS |"couNTERl 23 DEV CE EXTRACT'NG FEATURE FIG.1

FIG. FIG.

F I G. 1 B

24 Sheets-Sheet 2 SAWTOOTH START CONTROL Patented June 28, 1971 GK 5 mm n GATES m W N l SELECT ON 1 4 M PATTERN 6 A L m EL a G g H F W m [L N n n M R B g m 5 O 4 T fi W L C W E L d z GK 8 8 A E W f WL 1 WW N GATES W u M MT H SELECT ON MU B E L W M L PATTERN w T A L n M H V I X 4 L O S A a M k GATES E A MW 6 f SELECT ON w n u M I 1 W 4 r PATTERN F J mw MT A 8 UE 2 7%5 AA u T SN 7 \20 MW 5 1d 7 6 4 7. 6 33/.CIRCUIT m 5 5 4 8 2 5 7 M N f S M 1 GR 8 #2 NO URL :7 A MODULE M 5 T 6 A G T. C A. x S .1 5 AVO DANCE ME m I 4 J 5 CLOSED LOOP MEE E f 5 4 T8 3 9 R 6 A 9 e 6 (ill 5 8 Patented June 28, 1971 3,588,823

24 Sheets-Sheet 5 FIG.2B

Patented June 28, 1971 24 Sheets-Sheet 6 I I I 4k I X3 f I I I 49 QR D M J 24 46' -R X5 COMP A r I 21 535 N 3&5 L t MEM MA I 23 25 II R# o R I 4? x3 2 OR 3| I 4 I22 I 24 29 X6 COMP C I r 27 3 56 N 3 86 L2 I I MEM D/A FF- K31: I I 2 3 2 5 II R+ o I R I I g 47 20 I9 F i I I I 1 i I D M 2 J 46/OR R j X5 COMP k n I/ 3 21 7 3 I5I N 485 I MEM D/A V 21 I 4&5 "FF -31 I I 23 25 71 R" 0 R I II I I 14 20 4 l D i 34 x4 J I 46/0R FR 9 X6 COMP U} I 21 3;!6' 4 a 6 D/A F 25 II 0 8 R i I 20 77 I 49 70 I D x5 31 J 22 29 356 6/ X6 COMP W 27 1 1 N 5 8:6 21 MEM D/A I--CL sae 25 -31 FIG. 3C R I 41 I FIG. FIG. FIG. FIG. FIG. N PRESET BA 3D 36 3H 31 COUNTER 1 FIG FIG. FIG.

8 19 R 3 3E F|G.3

7 FIG FIG. FIG.

Patented June 28, 1971 3,588,823

24 Sheets-Sheet 8 FIG. 3E 31 I 523 H 324 35 Patented June 28, 1971 3,588,823

24 Sheets-Sheet 1O RECOGNITION "AND" GATES- (A) PATTERN TEACHING SELECTOR SUMMING NETWORK REGISTER (MAXIMUM DETECTOR SYSTEM) Patented June 28, 1971 3,588,823

24 ShetS-Shet 11 FIG.3H

RECOGNITION "AND" GATES-(B) l I I I I 51 I I I I l I PATTERN TEACHING SELECTOR v 812 gsm SUMMING NETWORK so 69 55B 72 REGISTER MAXIMUM DETECTOR SYSTEM 68 T0 OUTPUT DEVICE Patented June 28, 1971 3,588,823

24 Sheets-Sheet 18 H 2 3 14s 6 F|G.6

PATTERN1662 TEACHING'AAAAAAAAAAAAAAA 39 1&3 F

F 313 RECOG- 1&4 P HTIOQI AND 314 ATES 1&5

1&6

2&3

2&4

2&5

3&4

SUM MING NETWORK 5&6

4&5 34s 4&6

REGISTER MAXIMUM DET SYSTEM T0 OUTPUT DEVICE FROM 36 FIG. 3J 

